Methods and systems for improving maps

ABSTRACT

Methods, systems, and apparatus, including computer programs encoded on non-transitory computer storage medium(s), are directed to improving completeness of map information and data related to maps created through sensor data. Map completeness can be improved by determining object completeness and coverage completeness of a generated map and reducing amount of unknown areas of the generated map.

TECHNICAL FIELD

Various embodiments generally relate to improving maps.

BACKGROUND

Automated agents such as robots or cars rely on dynamic maps to navigatein a changing environment. Those maps are created from in-vehicle orremote (e.g., infrastructure) sensor detections and need to meet certainquality requirements in order to generate safe motion trajectories.While state-of-the-art multi-sensor fusion schemes entail a variety ofad hoc plausibility checks to verify the correctness and accuracy ofperformed sensor measurements, little attention is paid to theconsequences of the lack of information about specific objects or areas,i.e., the quality attribute of completeness. No-coverage regionsrepresent generic safety risks, as they may contain hidden movingobjects, and should, therefore, be avoided. Especially in the case ofremote infrastructure sensing, this is expected to be of high relevancesince undetectable areas may be located in the immediate driving path ofa vehicle. The problem remains, however, that an automated agent istypically not aware of the incompleteness of the map it uses, but onlyof the objects that were actually detected, except if a reference groundtruth is available (i.e., an alternative source of information for thesame environment). As a consequence, it cannot always make appropriatedecisions.

In general, to the extent there are previous solutions, there is noconcept for the systematical quantification of ignorance of the mapinformation. Ignorance about individual sensor measurements is used forthe purpose of data fusion, but no meaningful completeness measurerelated to the map information is reported to the end user. As aconsequence, previous completeness metrics are not considered fordecision-making of users. This is a safety-critical issue especially forcomplex automation environments such as a roadside sensorinfrastructure.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, like reference characters generally refer to the sameparts throughout the different views. The drawings are not necessarilyto scale, emphasis instead generally being placed upon illustrating theprinciples of the invention. In the following description, variousembodiments of the invention are described with reference to thefollowing drawings, in which:

FIGS. 1 and 2 show exemplary collaborative sensor fields in accordancewith or implemented in exemplary embodiments of the present disclosure.

FIG. 3 shows an exemplary method for improving completeness of a map inaccordance with or implemented in exemplary embodiments of the presentdisclosure.

FIG. 4 shows an exemplary process flow in accordance with or implementedin exemplary embodiments of the present disclosure.

FIGS. 5A-5D show further exemplary sensor fields in accordance with orimplemented in exemplary embodiments of the present disclosure.

FIG. 6 shows an exemplary process for determining and improving mapcompleteness in accordance with or implemented in exemplary embodimentsof the present disclosure.

FIG. 7 shows an exemplary process flow in accordance with or implementedin exemplary embodiments of the present disclosure.

FIGS. 8 and 9 show exemplary roadside sensor structures in accordancewith or implemented in exemplary embodiments of the present disclosure.

DESCRIPTION

The following detailed description refers to the accompanying drawingsthat show, by way of illustration, specific details and embodiments inwhich the invention may be practiced.

The word “exemplary” is used herein to mean “serving as an example,instance, or illustration”. Any embodiment or design described herein as“exemplary” is not necessarily to be construed as preferred oradvantageous over other embodiments or designs.

The words “plurality” and “multiple” in the description and claims referto a quantity greater than one. The terms “group,” “set,” “sequence,”and the like refer to a quantity equal to or greater than one. Any termexpressed in plural form that does not expressly state “plurality” or“multiple” similarly refers to a quantity equal to or greater than one.The term “lesser subset” refers to a subset of a set that contains lessthan all elements of the set. Any vector and/or matrix notation utilizedherein is exemplary in nature and is employed for purposes ofexplanation. Aspects of this disclosure described with vector and/ormatrix notation are not limited to being implemented with vectors and/ormatrices, and the associated processes and computations may be performedin an equivalent manner with sets or sequences of data or otherinformation.

As used herein, the terms “a” or “an” shall mean one or more than one.The term “another” is defined as a second or more. The terms “including”and/or “having” are open-ended (e.g., comprising). The term “and/or” asused herein is interpreted such that “A and/or B” means any of thefollowing: A alone; B alone; A and B. Similarly, A, B, and/or C meansany of the following: A; B; A and B; A and C; B and C; A and B and C.

As used herein, “memory” is understood as a non-transitorycomputer-readable medium in which data or information can be stored forretrieval. References to “memory” included herein may thus be understoodas referring to volatile or non-volatile memory, including random accessmemory (RAM), read-only memory (ROM), flash memory, solid-state storage,magnetic tape, hard disk drive, optical drive, among others, or anycombination thereof. Registers, shift registers, processor registers,data buffers, among others, are also embraced herein by the term memory.The term “software” refers to any type of executable instruction(s),including firmware, for example.

Unless explicitly specified, the term “transmit” encompasses both direct(point-to-point) and indirect transmission (via one or more intermediarypoints). Similarly, the term “receive” encompasses both direct andindirect reception. Furthermore, the terms “transmit”, “receive”,“communicate”, and other similar terms encompass both physicaltransmission (e.g., the transmission of radio signals) and logicaltransmission (e.g., the transmission of digital data over a logicalsoftware-level connection). For example, a processor or controller maytransmit or receive data over a software-level connection with anotherprocessor or controller in the form of radio signals, where the physicaltransmission and reception is handled by radio-layer components such asRF transceivers and antennas, and the logical transmission and receptionover the software-level connection is performed by the processors orcontrollers. The term “communicate” encompasses one or both oftransmitting and receiving, i.e., unidirectional or bidirectionalcommunication in one or both of the incoming and outgoing directions.The term “calculate” encompass both ‘direct’ calculations via amathematical expression/formula/relationship and ‘indirect’ calculationsvia lookup or hash tables and other array indexing or searchingoperations.

Various exemplary embodiments of the present disclosure relate tosystems, apparatuses, and/or methods related or directed to estimation(e.g., real-time estimation) of the completeness of a map (e.g.,dynamically created map) or a selected sub-region of the map,particularly in the absence of an external ground truth. In variousexemplary embodiments, the specifics of the sensor field design such as,but not limited to, positions, range, orientations, etc. are known. Thatis various exemplary embodiments herein relate to methods to estimatethe completeness of information for dynamic occupancy grid maps in theabsence of ground truth.

In one or more embodiments of the present disclosure, regions withsupervised borders, for example, confined road sections, may allow foran enhancement of the map completeness by object counting, and thusimprovement of the quality of information of the dynamic map.

FIGS. 1 and 2 show, according to exemplary embodiments of the presentdisclosure, examples of collaborative sensor fields. In FIG. 1, an areaof interest (AoI) 10 is covered by the field of views (FoVs) of thespatially distributed sensors 20 a, 20 b, 20 c, 20 d, and 20 e. In FIG.1, the FoVs 30 a, 30 b, 30 c, 30 d, and 30 e do not wholly cover theAoI. Each sensor sends or transmits its sensor data to a centralcomputation node 50. In some embodiment, the central computation node isknown as a fog node, or simply the fog. Further, each sensor may becharacterized by its FoV, range, resolution, orientation, sensorlocation, and can have a unique sensor index. In various embodiments,this or related sensor information may be known to the fog 50 at anyinstant of time (so if necessary it is dynamically updated).

In general, sensors described herein may transmit sensor data to acomputation node (e.g., central computation node) or fog through anysuitable means, e.g., directly or indirectly through wired or wirelesslymeans. A central computation node may be any suitable computing deviceor devices. Further, in accordance with various embodiments of thepresent disclosure, sensor detection information (e.g., target position,target velocity, size, etc.) may be reported or electronically conveyed(e.g., wired or wireless) from a sensor to the central node. This sensorinformation may be repeatedly and/or continuously transmitted to the fogover time.

In general, the sensors described herein may be radar sensors, cameradevices, light, LIDAR sensors, and/or any other suitable sensors (e.g.,a sonar sensor, a LIDAR sensor, radar sensor, a video/camera imagesensor, or a V2X sensor). For example, one sensor on a vehicle (e.g., amotor vehicle) may be a mounted rotating LIDAR sensor.

In various situations, for the sake of simplicity, it may be assumedthat the probability of a false positive or a false negative detectionis negligible if a sensor has a direct line of sight to the target. Atarget is missed by the sensor field if it is occluded by other objects,or not in the sensor's FoV. Further, in some exemplary embodiments,false positive measurements can be eliminated to a significant extent byusing tracking algorithms. For example, the chance for a false negativedetection of an object in direct sight of a sensor can depend on thetechnical capabilities of the sensor but is usually small. In variousembodiments, sensors are used that have the capability to detect or readthe size of a detected object.

A computing node (e.g., central computing node) or fog 50 may include orbe one or more computing devices which, among other things may beconfigured to dynamically generate a map using sensor data obtained fromsensors. More specifically, the central node or fog may implement afusion mapping algorithm to dynamically generate a map from obtainedsensor data. That is, the fog hosts computational processes to fuseindividual sensor data in order to create a dynamic map. The map may bedynamically created or updated in real-time. In various embodiments,occupancy grid maps (OGM) may be built from uniformly spaced cells as adiscretized environment model.

In addition to dynamically creating a map with received sensor data, thefog may be further configured to monitor the health of sensors, e.g.,via a heartbeat update signal, and be configured to immediate detect orrealize sensor outages.

In further exemplary embodiments, the map information (e.g., thegenerated map) may be transmitted (e.g., via wired or wirelesstechnology) to an agent for the purpose of collaborative awareness andnavigation. The agent may be a process or algorithm implemented by thefog or another computing device.

In one example, a vehicle may use the map input in order to deduce adriving decision. That is, agents may have to be able to associate their“ego perspective” with the map content for self-localization, i.e., theyshare a reference coordinate system with the sensor field. This is, forinstance, the case if both the agents and the sensor field possess amodule for GNSS positioning (global navigation satellite system).

In various embodiments of the present disclosure, an area of interest(AoI) is a well-defined area (or areas in certain circumstances) inspace. An AoI may be determined by a user (e.g., user defined by userinput) and/or may encompass a relevant region for a task at hand. An AoImay, but does not necessarily, overlap with the area surveilled by acollaborative sensor field. For instance, the AoI may be a sub-region ofa map that is of particular interest for an imminent driving operation.If the AoI has only a small or no overlap with the extent of the dynamicmap, this presents substantial design incompleteness.

Referring back to FIG. 1, shows the AoI 10 which is surveilled by thesensors by sensors 20 a, 20 b, 20 c, 20 d, and 20 e. The sensors 20(sensors 20 a, 20 b, 20 c, 20 d, and 20 e) of FIG. 1 may be implementedas part of a roadside sensor infrastructure that is used to supportautomated roads. However, the FOVs 30 a-30 e do not entirely overlapwith or cover the AOI 10.

In the example of FIG. 2, the sensors and fog are included or integratedwith vehicle 40. That is, the vehicle 40 has onboard sensors forenvironment perception. In the case of FIG. 2, the AoI is not fixed orconfined. In FIG. 2, the sensors have FoVs 30 a, 30 b, 30 c, 30 d, and30 e which are fixed relative to the vehicle 40. The AoI in this exampleis an area in front of the vehicle 40.

FIG. 3, shows according to at least one exemplary embodiment of thepresent disclosure, an exemplary method for improving the completenessof a dynamically created map. FIG. 3 may be understood in conjunctionwith the exemplary embodiment depicted in FIG. 4. That is FIG. 4 shows,according to at least one exemplary embodiment, a process flow relatedto the method of FIG. 3. The method depicted in FIG. 3 may beimplemented by one or processors of one or more computing devices. Theone or more computing devices may be a central node or fog.

At 310 of FIG. 3, one or more computing devices obtain sensor data orsensor information from a plurality of sensors over time, wherein theplurality of sensors define a sensor field. For example, the sensors maybe spatially distributed as shown in FIGS. 1 and 2.

At 320 of FIG. 3, the one or more computing devices dynamically generatea map by fusing the obtained sensor data, wherein the generated mapincludes a plurality of grid cells at least partially covered by thesensor field. That is, the generated map may be or may includeinformation representing or regarding the external environment coveredby the sensors, e.g., each grid cell may include or may be associatedwith external or environmental information. The grid cells or simplycells may be any suitable shape or size and may depend on thecharacteristics of the sensors. That is, the grid cells are notnecessarily uniform in shape or size. Any suitable sensor fusionalgorithm may be used to create a sensor map or mapping from the sensordata, including but not limited algorithms/techniques involving orapplying Central limit theorem, Kalman filter, Bayesian networks,Dempster-Shafer, Convolutional neural network, etc.

Further in one or more embodiments, the exemplary method of FIG. 3 mayfurther include receiving an AoI, e.g., from a user. Accordingly, thegenerated map may include or be limited to the AoI, which may be asubregion of a subsection of the map. FIG. 4, shows the flow of sensordata or sensor info 405 into the fog 450 in order to generate a map at410. FIG. 4 shows at 415 input may be received, e.g., from a user or anelectronic source, which specifies an AoI 415. The AoI is transmitted orobtained by the fog 415 and used to update or refine the map 410 to anAoI map 420.

Referring back to FIG. 3, at 330, the one or more computing devicesdetermine completeness of the generated map by determining objectcompleteness and coverage completeness of the generated map from theobtained sensor data. In one or more embodiments, the one or morecomputing devices may determine the completeness of a subsection orsubregion of the generated map, e.g., the AoI. FIG. 4 shows the sensorinfo 405 used by the fog 450 to determine the map completeness bycalculating unknowns at 425. The completeness determination, asexplained, in more detail below, may include determining unknown areasof the map. For example, the unknown areas may be areas (e.g., cells)wherein the occupancy status is unknown or not ascertainable by lack ofsufficient sensor information for that area.

At 340 of FIG. 3, the one or more computing devices may update (orimprove) completeness of the generated map by eliminating unknown areasof the generated map using the determined object completeness and thedetermined coverage completeness. In other words, the map or the mapinformation may be updated to reduce the amount of or number of unknownareas. That is, one or more unknown areas of the map may now beconsidered as “known”. For example, the one or more computing devicescan use the completeness information to eliminate or reduce the amountof or number of unknown areas in an obtained AoI, rather than reduceunknown areas in the entire map.

For example, FIG. 4 shows the map of AOI with unknowns being updated at430 and the completeness information is also determined or calculated at435. This updated information may be further used. For example, theinformation associated with determining completeness may includecalculation of completeness metrics (e.g., Γ(AoI). Thus the updated map(with reduced unknown areas) and/or the completeness metrics may betransmitted or used as input in a further decision process at 440. Inone example, the updated AOI map and the completeness metrics may beinputted into a self-driving algorithm or any other computer processwhich uses the data. In some embodiments, the information is simplygiven or presented (e.g., visually) to the user, e.g., through a displaydevice so as to be relied on by the user.

In general, map completeness reflects the degree to which the map orsensor is sufficient to represent the universe of discourse (e.g., AoI).Map completeness is generally reduced or lowered by the failure ofcapturing a portion of information of the ground truth, e.g., by asensor missing an object (e.g., a passing object). Referring back toFIG. 3, at 330, two types of completeness are determined or calculated,namely, object completeness and coverage completeness. FIG. 6, showsaccording to at least one exemplary embodiment, a process fordetermining and improving map completeness by determining object andcoverage completeness.

In general, in an object-based environment representation model, objectcompleteness can be measured as the ratio of objects (includingattributes such as size, position, and velocity) reported in a map(e.g., AoI map) and objects according to the ground truth. That is, theobjects that are known are compared to the objects that are currentlypresent. However, an incomplete object number does not yield informationabout the location of the missing objects. Furthermore, the respectiveground truth is typically not available in a realistic dynamicenvironment.

In a grid-based environment model, on the other hand, the unit ofinformation is not an object entity but instead a grid cell (a Euclideanarea) with a known or unknown status. In this case, coveragecompleteness may be determined or calculated by comparing the numbers ofgrid cells that are known with the number of the grid cells that arecurrently present. This is equivalent to the ratio of the spatiallycovered part of the AoI and the full extent of the AoI.

Coverage and object completeness may be represented, in general, bydifferent metrics and will coincide only in the case of perfectlyhomogeneous traffic, and minimal cell occlusions. This can be understoodas follows: For the ideal case of an ideally homogeneous vehicledistribution having a constant density of vehicles per area, in both thecovered and uncovered sections of the AoI, the coverage completeness andobject completeness measures will, on average, even out or coincide toeach other. However, a passing object may cast a shadow on grid cellsbehind an area that—depending on the sensor field design—might be quitedifferent from the area corresponding to the occluded vehicles in thisshadowed region. Therefore, both measures can differ in practice.

As discussed in FIG. 3, completeness of a generated map is improved byeliminating unknown areas using both the determined object completenessand the determined coverage completeness.

In various embodiments of the present application, object completenessmay include determining whether any objects are missing from a map,e.g., a AoI. The map, as, described in various embodiments, may begenerated through sensor fusion e.g., an implementation of a sensorfusion algorithm that operates on obtained sensor data. Sensor fusionalgorithms generally includes methods, techniques, or processescombining of data from several sensors or sources for the purpose ofimproving application or system performance. (See e.g.,https://en.wikipedia.org/wiki/Sensor_fusion). In various exemplaryembodiments, sensor data may be of environmental nature from varioussources that is intelligently or merged or combined for localization andmapping of outdoor or external environments.

For example, FIG. 6 at 610, one or more computing devices (e.g., a fog)is configured to determine objects entering and leaving at least oneportion of the generated map and determine a current number of objectsin the at least one portion of the generated map. This may beaccomplished by monitoring of object flow in and out of the map area(AoI). In other words, the entry and exit areas of the map or AoI may besupervised so that objects cannot enter or exit unnoticed. Sensors,which may be in addition to and independent of the sensors used togenerate a fusion map, may be deployed for monitoring or supervising theingress and egress of objects. Such additional sensors, for example, mayinclude cameras, light barriers, or devices implementing wirelesshandshake messages at the entry and exit points.

In short, the fog may receive the sensor information to dynamically orcontinuously track and concurrently count the number of objects leavingand entering as well as the number of objects currently in the AoI. Thecount of objects may be done or implemented at any instant of time toestablish a dynamic ground truth with respect to the number of objects.As a prerequisite, the ground truth may need to be calibrated once,e.g., by a blank or empty scene.

Furthermore, determining object completeness may include separately orindependently monitoring or tracking each of the objects in the map orAoI. At 620 of FIG. 6 the fog tracks or monitors each of one or moreobjects in the at least one portion of the generated map. The trackingof the objects may be done by the collaboration of sensors making ordefining the sensor field. In this case, these sensors can detect thepresence of objects and then be used to track the movement of objects.Also, the sensors can be used to detect and record various attributes ofthe objects in the sensor field including, for example, target position,target velocity, size, orientation, etc. This sensor information isreported or transmitted to the fog which uses the information to trackeach object. In various embodiments of the present disclosure, thissensor data is associated with each object, e.g., the detection (e.g.,where, when, etc., an object is detected) as well as the objectattributes (size, position, velocity, orientation, etc.) can be storedin any suitable computer storage medium, e.g., one or more databaseoperatively connected to the fog. Furthermore, the statuses of thesensors (sensor operating status, sensor location, sensor FoV, etc.) mayalso be stored. Upon detection of an object, the fog may uniquely assignan identification to the tracked object (which can be stored with theassociated attributes of the tracked objects). In short, the fog may atany time retrieve rely on such information, e.g., the historical datasensor data.

In FIG. 6, at 630, the fog may determine object completeness of at leastone portion of the generated map. That is, based on the verifying of thedetermined current quantity of objects (in the AoI) and the number ofobjects that have left or entered the AoI with the tracking, the one ormore computing devices can determine the extent of object completeness.For example, there will be object completeness when the object trackingdoes not detect the disappearance of any objects and the trackingcorresponds with the amount or quantity of current objects factoring inthe number of objects that have entered or left. That is, based on thecurrent quantity and amount objects entering and leaving, the trackinginformation can indicate whether any objects are missing. If there areno objects missing, for example, if none of the tracked objects goes“missing” and the net change of objects in the AoI is accounted for bythe objects that entered or left, then the map can be determined to becomplete. That is, a map determined to be complete means no object isunaccounted for and thus there are no unknown grid cells with respect tooccupancy in at least the AoI of the map. Then the map can be updated atstep 660.

In the case where there are no missing objects—then there would beconsidered no unknown areas—e.g., no areas where the occupancy of theareas (grid cell) is unknown.

However, in the case where a map is incomplete, e.g., one or moreobjects are missing or unaccounted for, then at 640, a prediction spaceis determined. For example, when one or more objects are determined tobe missing, e.g., based on analyzing the tracking and count information(current quantity information and number of objects the entering andexiting), then the fog determines or calculates the prediction space foreach missing object. The prediction space of all missing objects is theunified prediction space.

In accordance with exemplary embodiments of present disclosure, variousevents associated with an object may be detected using the sensors. Forexample, the fog may use obtained sensor data to detect or identify adisappearance event as discussed. Disappearance of an object may beascertained by comparing the detection history in the preceding timestep(s) to the detection(s) in the current time step so that the absenceof a steady continuation of the vehicle path can be verified. Thus, thefog can infer that the object, e.g., a vehicle has either left the FoVof the sensor field or is occluded.

In various embodiments, the fog uses obtained sensor data to determinean exit event. An exit event may be equivalent to a disappearance eventexcept for the fact that that the vehicle has left the map (AoI) in thespecified time interval. The fog may be configured to determine oridentify an entry event. An entry event may be equivalent or similar toan exit event except for the fact that the vehicle has entered the AoIin the specified time interval. The fog may be configured to determineor ascertain an appearance event. An appearance of an object may beascertained or discovered by comparing the detection history in apreceding time step(s) to detection(s) in the current time step. Vehicledetections occur for vehicles that have no continuous track history. Thefog can infer or determine that an object has now or just entered asensor FoV after it was occluded, or out of the FoV a time step ago.

Upon realizing an appearance event, the fog can cancel the predictionspace associated with the reappeared object in order to free theinvolved grid space. Further, the fog can reassign the object ID tospecify which of the previously occluded objects has reappeared. As morethan one object can be located within the prediction space, thereassignment might not always be unique, in which case a portion of theprediction space should be kept. Reassignment is facilitated if,together with the ID, the object type is also registered and stored.

In various exemplary embodiments of the present disclosure, objectstatus is managed to adapt to the above events. Below is exemplarypseudocode that illustrates such logic. Two lists are maintainedfeaturing all objects in the AoI, at subsequent time steps (previous andcurrent):

  list_prev={v_1,v_2, ...};   list_curr={v_1, v_2, ...}; Each vehicleobject is a structure with at least the following properties  v=struct(ID, position, velocity, size, status, prediction space);  status ϵ {vis, invis, noAoI}; The status can be either of visible(vis), invisible (invis) or not in the AoI (noAoI).   Initializelist_prev, list_curr;   WHILE time<time limit       FOR all detections      Write list_curr.position, list_curr.velocity, list_curr.size,      list_curr.status=vis or noAoI;       ENDFOR   Assign list_curr IDsby matching positions, velocities of list_curr to   list_prev;  Complement list_curr with items present in list_prev but not in  list_curr (so list_curr.status=invis);   Identifyexit/entry/appearance/disappearance + continuing invisibility   eventsfrom list_curr and list_prev status;       FOR all events          IFdisappearance or continuing invisibility          Update predictionspace using last known position          and velocity in list_prev;         ELSEIF appearance event          Reassign ID and resetrespective prediction space;          ELSEIF exit event          Deletefrom list_curr;          ENDIF       ENDFOR   time=time+1;  list_prev=list_curr;   ENDWHILE

Regarding coverage completeness, a map (e.g., a map generated through afusion algorithm based on collaborative sensor data as described herein)may include a plurality of grid cells. Referring again to FIG. 6, at 650coverage completeness is determined by computing devices/fog throughcalculating or determining unknown grid cells of the generated map orthe AoI of the generated map. In other words, to determine coveragecompleteness, the state of each grid cell is evaluated. In at least oneembodiment of the present disclosure, the computing devices determinewhether or not there is sufficient sensor information for each grid cellto determine whether the particular cell is occupied or not. Moreover,the sensor information is evaluated whether the sensor information isadequate to establish whether or not the occupancy state of the gridcell is known. For example, if there is no sensor information or thesensor information is inadequate for a particular grid cell, then thatparticular grid cell is determined to be and/or assigned an “unknown”state. For example, the state of “unknown” can be invoked or determinedif no sensor information for the grid cell is available because the cellis not covered by the sensor field.

By contrast, if the sensor information of that grid cell is sufficientto establish the occupancy state, then the grid cell is determinedand/or assigned a “known” state. Further, in the case of the grid cellstate being known, the grid cell can be further determined and/orassigned as having an “occupied” or “unoccupied” state. That is, if thesensor data indicates there is at least one object in the grid cell,then the grid cell can be determined or assigned an occupied state.Similarly, if the sensor data indicates there is no object in the gridcell, then the grid cell can be determined or assigned an “unoccupied”state. In accordance with exemplary embodiments, the object can bealmost anything that occupies a grid cell. In some embodiments where thegenerated fusion map or AoI includes a road, the object(s) may be amotor vehicle, bicycle, pedestrian, animal, rock, etc.

In the present disclosure, the unified set of all grid cells with thelabel unknown may be denoted S_(unknown). The “unknownness” of a gridcell may be attributed to different sources such as design completeness,systematic completeness, and sensing-related completeness.

Unknown grid cells due to design completeness, denoted S_(design), occurwhen a portion of the map (e.g., AoI) is not covered by the sensor fielddue to design constraints. Unknown grid cells due to systematiccompleteness, denoted S_(failed), occurs when a portion of the map (AoI)is currently not covered by the sensor field due to the failure of oneor more sensors, but would otherwise be covered if the sensor field wasfully functional. Unknown grid cells due to sensing-relatedcompleteness, denoted S_(occluded), occurs when a portion of the map(AoI) that is currently not covered by the sensor field even though thesystem is working properly or correctly. For example, this may be causedby an object occluding or blocking of sensing of grid cells.

In accordance with various exemplary embodiments of the presentdisclosure, the parameters S_(design), Sf_(ailed), S_(occluded), andS_(unknown) may be depicted in FIGS. 5A-5D. As shown, FIGS. 5A-5Dinclude the sensors, sensor field, AoI, etc. depicted in FIG. 2. Furtherincluded in these figures is a plurality of objects 60, which in thisexample are vehicles within the AoI.

The one or more grid cells belonging to the set S_(design) can bedetermined or ascertained directly by comparing the known sensor FoVsand orientations to the map or AoI. S_(design) can change due to shiftsof the area of interest, or by a reconfiguration of the sensor field. Inthe example of FIG. 5A, the cells 80 a are cells that are unknown due todesign constraints and thus belong to S_(design). As shown, the cells 80a are not covered by the sensors, e.g., outside the FoV of the sensors,but are cells in the AoI.

The one or more grid cells belonging to the set S_(failed) can bedetermined or derived by locating a failed sensor in the sensor field byits unique ID, and analyzing the change in coverage of the map or AoI.This can be readily achieved in the case that the configuration of eachsensor is known to the fog node. S_(failed) can vary on the typicallyvery large timescale of the order of the mean lifetime of a sensor, orif the map or AoI is modified. In the example of FIG. 5B, the cells 80 bare cells that are unknown due to sensor failure and thus belonging toS_(design). As shown, the cells 80 b are within the field of view offailed sensor 20 b. In other words, these cells are in the AoI but andwithin the designed sensor filed but are unknown due to sensor failure.

The one or more grid cells belonging to S_(occluded) can be ascertainedor determined based on object detections made by the sensor field. Forexample, the fog node can project the subdomains of the map or AoI thatare shadowed behind detected objects while taking into accountmeasurement uncertainties. This set changes continuously, following theobject motion. Sensing-related completeness may be particularlyimportant for a roadside sensor infrastructure, as the sensingperspective might be quite different from the perspective of a vehicleon the road, and occluded areas can therefore occur in immediateproximity of moving objects. In the example of FIG. 5C, the cells 80 care cells that are unknown due to occlusion or blocking sensor failureand thus belonging to S_(design). As shown, the cells 80 c are withinthe field of view of the some of the sensors but are blocked by theobjects (vehicles) 60 c.

Finally, FIG. 5D shows the total of unknown cells due to designconstraints (S_(design)), sensor failure (S_(failed)), and occlusion(S_(occluded)). Accordingly, the total set of unknown cells then is theunion, based on coverage completeness approach is:S_(unknown)=S_(design)∪S_(failed)∪S_(occluded)

Therefore, the overall coverage completeness Γ of the map or AoI withoutfurther modification is:

$\Gamma = \frac{S_{unknown}}{S}$where S is the set of all grid cells including the entire area ofinterest. F usually refers to a chosen AoI of a map.

For an arbitrary traffic scenario and any AoI, the coverage completenessdefined above can serve as a quality measure for the usability of thedynamic map. A general estimate of this kind will lead to upper-boundincompleteness values, and thus to overly cautious decisions, since alloccluded areas are treated as zones of safety risks. Further, coveragecompleteness is determinable even in the absence of an object numberground truth, and may therefore be especially useful in very dynamicenvironments of low predictability, such as a densely populated urbansetup.

In accordance with various embodiments the fog or central computing nodecan be configured to determine the total set of unknown cells and/or thetotal completeness. The fog may determine this information dynamically,e.g., the information is determined or calculated as the obtained sensorinformation is received or updated.

In accordance with exemplary embodiments of the present disclosure, if afog node realizes a disappearance event, it may be triggered tocalculate an individual prediction space for the respective object(e.g., vehicle). For example, a prediction scheme may be implemented bythe fog so as to estimate the set of all grid cells that can bephysically occupied. Accordingly, a disappearance event is realized,e.g., from the fog using the senor data, then the fog may resort to anduse the immediate object history in order to determine or estimate theobject horizon, e.g., the possible positions (grid cells) the object canoccupy. In various embodiments, determination of the prediction spacemay rely on using a mechanical motion model of the disappeared objectbased on one or more last known positions and velocities. Any suitablealgorithm or process may be used to estimate the object horizons. Forexample, in the case that the objects are vehicles, a maximal steeringangle and physical acceleration rate may be used by the fog to determinethe vehicle horizons.

The set of grid cells within the union of all such object horizons maydenoted the unified prediction space S_(physical). In other word,S_(physical) includes the possible positions of missing objects, e.g.,includes the one or more grid cells that which can be occupied by one ormore missing objects.

Back in FIG. 6, at 660 the computing devices or fog update the mapcompleteness by reducing the amount of unknown grid cell. For example,in accordance with exemplary embodiments, after the object completenessdetermined, (e.g., S_(physical) is determined) then coveragecompleteness can then be calculated and improved. Updating mapcompleteness can be implemented by eliminating as “unknown” grid orre-identifying or reassigning certain identified unknown grid cells to“known”. In particular, the unknown grid cells (e.g., S_(unknown)) thatare NOT members of the unified prediction space (e.g., S_(physical)) canbe eliminated. These are cells are the ones that would not be occupied.Thus, the new or updated cells are those cells belonging to theprediction space that also overlap with the originally determined orcalculated unknown cells.

In the case where all objects are accounted for (full objectcompleteness) it is therefore realized by the computing devices thatthese original unknown cells are not be occupied by any objects. Bycontrast, in the case where there is a disappearance event—grid cellsthat were originally unknown can be eliminated as unknown because theprediction space indicates that the “missing objects” are or would notbe in these grid cells.

In exemplary embodiments, the fog can use the unified prediction spaceto eliminate unknown grid cells from the previous set of unknown gridcells. This refinement by the fog may be expressed as follows:S_(unknown)′=S_(unknown)∩S_(physical)S_(unknown)′=S_(design)∪S_(failed)∪S_(occluded)∩S_(physical)

In short, the S_(unknown)′ is the intersection of the previousS_(unknown) and S_(physical).

Depending on the system design, this refinement can significantlyimprove the completeness quality measure, as is demonstrated in the nextsection. The corner case of a complete object detection is important: Ifthe presence of all objects is verified—there are no disappearanceevents and thus there are no missing objects—then all unknown grid cellscan safely be considered as unoccupied, leading to a temporarilycomplete dynamic map.

The elimination of unknown grid domains is possible only because theobject number ground truth verifies the absence of objects in theseareas. If the entry and exit points of the AoI are not monitored by thesensor field, S_(physical) can be evaluated as well, however this doesnot help the coverage completeness. The possibility of an objectappearing sometime somewhere in a temporarily occluded domain cannot beexcluded (e.g., a pedestrian may step from the sidewalk on the road).Therefore, even if it is known that a vehicle just travelled from amonitored area into a shadowed area, predicting the object or vehiclehorizon does not provide any useful insight.

FIG. 7, shows, according to at least one exemplary embodiment of thepresent disclosure, the flow of FIG. 4 updated with the calculateunknown section updated as described herein.

FIG. 8 shows, according to at least one exemplary embodiment of thepresent disclosure, an exemplary roadside sensor structure of a roadsegment 800. The fog may implement a method to dynamically improve mapcompleteness in accordance with exemplary embodiments.

The roadside sensor structure may include a plurality of sensors 805that are operatively communicative with the fog to monitor a roadsegment (AoI) 800 which currently includes cars 870 a, 870 b, 875, andtruck 880. The field of views of the sensors may overlap so as toprovide particular emphasis or redundant surveillance at predefinedentry 810 a and exit points 810 b. Through detection of all incoming andoutgoing objects, the sensor field can establish a notion of groundtruth with respect to the number of vehicles in the AoI.

As shown in FIG. 8, one car 875 has just entered an occluded area behinda truck 880. The sensor field can be used to realize a temporary objectincompleteness (e.g., the disappearance of the car 875). The fog, inresponse can, based on using and analyzing the most recent detectionhistory, determine the domain of physically possible positions of themissing car. The intersection of this set of cells with all unknowncells defines the dynamic coverage incompleteness of the AoI, whichrepresents a quality measure of the AoI map. The light cells 820 showcells that from coverage completeness analysis are determined be known,that is the occupancy statuses of these cells is originally known.Similarly, the darker cells 830 are cells that from coveragecompleteness analysis are determined to be unknown. For example, sensor805 f of the sensors 805 in FIG. 8 is a failed sensor, therefore thecells 830 f are unknown due to this failed sensor 805 f.

In FIG. 8, the dark cells 840 are originally part of unknown cells 830.In other words, the dark cells 840 are originally part of unknown cells830 due to occlusion caused by the truck 880. In response to the fogrealizing, through the sensor tracking of the car 875, that the car 875has disappeared, the fog immediately accesses and uses past orhistorical sensor data in implementing a prediction scheme to estimateor determine the possible positions of the car 875. In this case, thedark cells 840 are the cells in which the cells determined from theprediction space intersect or overlap with the original unknown cells830. Accordingly, the remaining cells that are outside or not part of840 can all be identified and considered as having a known occupancystatus e.g., unoccupied.

In exemplary embodiments, the generated map with updated completenessalong with completeness metrics may be used by other agents (human orcomputer) in order to make decisions. For example, FIG. 9 shows,according to at least one exemplary embodiment of the presentdisclosure, an exemplary roadside sensor structure of a road segment800. The fog may implement a method to dynamically improve mapcompleteness in accordance with exemplary embodiments. Similar to theroad segment of FIG. 8, the road segment of FIG. 9 may include roadsidesensors 905 that are operatively in communication with the fog. Inaddition, the vehicles 910 a and 920 b also include sensors (not shown).

In this case, the road segment is a two-lane highway in which the carnumber 910 a is blocked by a slow vehicle, truck 920, that it wishes toovertake. The AoI for car 910 a to undertake such a maneuver extends farback because fast vehicles, such as vehicle 910 b on the left lane, haveto be anticipated in order to make a safe driving decision. However, theonboard sensing range of vehicle 910 a in this case is not sufficient tocover this area. Therefore, the respective candidate map of the AoI forthe vehicle 910 a is highly incomplete (by design). However, theroadside infrastructure is capable of providing an almost completedynamic map of the AoI, and thus verifies that overtaking is currently asafe option. In this example, both sources of information do not detectany immediate threats in the AoI while the onboard sensing field has noevidence at all. However, an agent, with both sources of information,e.g., dynamic maps with improved or updated completeness information,that is using the infrastructure sensor to actively verifies the absenceof a safety risk can improve decisions. Thus a decision is therefore notbased on collision avoidance, but on a completeness measure.

In various embodiments of the present disclosure, ground truth (e.g.,the number of all possibly visible grid cells) is inferred given thatthe extent of the particular AoI is known, in contrast to the objectnumber ground truth. Uncovered domains of the grid may only give a hintfor possible object locations, but do not necessarily have to containany missing objects, and thus the resulting completeness measure isoverly conservative.

The exemplary embodiments of the present disclosure may be realized bycomputing device(s) performing the methods or similar methods describedherein. For example, a computing device may include one or moreprocessors configured to execute instructions (e.g., computer/hardwareinstructions) stored on and operatively accessible from suitablenon-transitory computer readable media. Thus processor(s) of theterminal device can execute the instructions, which to cause thecomputing device to implement the methods or variations of the methodsdiscussed herein.

While the above descriptions used various exemplary use cases, the useof these specific examples serve to enhance the clarity of thedescription and do not limit the applicability or scope of thetechniques described herein. While the above descriptions and connected,figures may depict electronic device components as separate elements,skilled persons will appreciate the various possibilities to combine orintegrate discrete elements into a single element. Such may includecombining two or more circuits for form a single circuit, mounting twoor more circuits onto a common chip or chassis to form an integratedelement, executing discrete software components on a common processorcore, etc. Conversely, skilled persons will recognize the possibility toseparate a single element into two or more discrete elements, such assplitting a single circuit into two or more separate circuits,separating a chip or chassis into discrete elements originally providedthereon, separating a software component into two or more sections andexecuting each on a separate processor core, etc.

The following examples pertain to further aspects of this disclosure:

Example 1 is a method for execution by one or more computing devicesincluding obtaining sensor data from a plurality of sensors over time,the plurality of sensors covering a sensor field; generating a map byfusing the obtained sensor data, wherein the generated map comprises aplurality of grid cells at least partially covered by the sensor field;determining, using the obtained sensor data, completeness of at leastone portion of the generated map by determining object completeness andcoverage completeness of the map from the obtained sensor data; andupdating completeness of at least the portion of the generated map byreducing an amount of unknown areas of the at least one portion of thegenerated map using the determined object completeness and thedetermined coverage completeness.

In Example 2, the subject matter of Example 1, wherein determiningobject completeness may include: obtaining sensor data from ingress andegress sensors; determining, from data obtained from ingress and egresssensors, objects entering and leaving at least one portion of thegenerated map; determining, from the obtained sensor data, a currentquantity of objects in the at least one portion of the generated map;and tracking, from the obtained sensor data, each of one or more objectsin the at least one portion of the generated map.

In Example 3, the subject matter of Example 2, wherein determiningobject completeness may include determining disappearance of one or moreobjects from the at least portion of the generated map based on thetracking, the determined amount of objects and a net change of objectsin the at least one portion of the generated map.

In Example 4, the subject matter of Example 3, wherein in response todetermining one or more objects has disappeared from the at least oneportion of the generated map, the method may further include determininga prediction space, wherein the prediction space comprises a set of gridcells for which the one or more determined disappeared objects cancurrently occupy.

In Example 5, the subject matter of Example 4, wherein determining theprediction space may further include: for each determined disappearedobject, determining possible positions the disappearing object cancurrently occupy using past sensor data of each of the disappearedobjects.

In Example 6, the subject matter of Example 5 wherein determiningcoverage completeness can include determining one or more unknown gridcells of the generated map, wherein the one or more unknown grid cellsare grid cells for which there is insufficient sensor information.

In Example 7, the subject matter of Example 6, wherein updatingcompleteness of at least the portion of the generated map can includeeliminating unknown grid cells that are members of the determinedprediction space.

In Example 8, the subject matter of Example 7, wherein eliminatingunknown grid cells can include assigning a known status to theeliminated unknown grid cells.

In Example 9, the subject matter of any of Examples 6 to 8, whereindetermining coverage completeness can further include determining one ormore known grid cells of the generated map, wherein a grid cell is knownin response to determining from the obtained sensor data that the gridcell is occupied or unoccupied by an object.

In Example 10, the subject matter of any of Examples 6 to 9, wherein oneor more grid cells can be determined to be unknown in response todetermining the one or more grid cells are not covered by the sensorfield.

In Example 11, the subject matter of any of Examples 6 to 10, whereinone or more grid cells can be determined to be unknown in response todetermining the one or more grid cells are not covered due to failure ofone or more of the plurality of sensors.

In Example 12, the subject matter of any of Examples 6 to 11, whereinone or more grid cells can be determined to be unknown in response todetermining the one or more grid cells are not covered due to occlusion.

In Example 13, the subject matter of any of Examples 2 to 12, whereinthe object can be a vehicle.

In Example 14, the subject matter of any of Examples 1 to 13, whereinthe method may further include obtaining an area of interest, whereinthe at least one portion of the generated map is the obtained area ofinterest.

In Example 15, the subject matter of any of Examples 1 to 14, whereinthe method may further include determining one or more completenessmetrics of the at least one portion of the generated after improving thecompleteness.

In Example 16, the subject matter of any of Examples 2 to 15, whereindetermining object completeness may include determining no objects aremissing from the at least portion of the generated map based on thetracking, the determined amount of objects and a net change of objectsin the at least one portion of the generated map.

In Example 17, the subject matter of Example 16, wherein in response todetermining no objects are missing, the method may include determiningall grid cells of the least one portion are known.

Example 18 is one or more computing devices including one or moreprocessors and at least one non-transitory computer-readable storagemedium that include instructions that, when executed by the one or moreprocessors, cause the one or more processors to obtain sensor data froma plurality of sensors over time, the plurality of sensors covering asensor field; generate a map by fusing the obtained sensor data, whereinthe generated map comprises a plurality of grid cells at least partiallycovered by the sensor field; determine, using the obtained sensor data,completeness of at least one portion of the generated map by determiningobject completeness and coverage completeness of the map from theobtained sensor data; and update completeness of at least the portion ofthe generated map by reducing amount of unknown areas of the at leastone portion of the generated map using the determined objectcompleteness and the determined coverage completeness.

In Example 19, the subject matter of Example 18, wherein the executedinstructions can cause the one or more processors to determine to objectcompleteness by causing the one or more processors to: obtain sensordata from ingress and egress sensors; determine, from data obtained fromingress and egress sensors, objects entering and leaving at least oneportion of the generated map; determine, from the obtained sensor data,a current quantity of objects in the at least one portion of thegenerated map; and track, from the obtained sensor data, each of one ormore objects in the at least one portion of the generated map.

In Example 20, the subject matter of Example 19, wherein the executedinstructions can cause the one or more processors to determine to objectcompleteness by causing the one or more processors to determinedisappearance of one or more objects from the at least portion of thegenerated map based on the tracked objects, the determined amount ofobjects, and a net change of objects in the at least one portion of thegenerated map.

In Example 21, the subject matter of Example 20, wherein in response tocausing the or more processors to determine one or more objects hasdisappeared from the at least one portion of the generated map, theexecuted instructions can further cause the one or more processors to:determine a prediction space, wherein the prediction space comprises aset of grid cells for which the one or more determined disappearedobjects can currently occupy.

In Example 22, the subject matter of Example 21, wherein the executedinstructions can cause the one or more processors to determine theprediction space by further causing the one or more processors to: foreach determined disappeared object, determine possible positions thedisappearing object can currently occupy using past sensor data of eachof the disappeared objects.

In Example 23, the subject matter of Example 22, wherein the executedinstructions causing the one or more processors to determine coveragecompleteness can further include the executed instructions causing theone or more processors to: determine one or more unknown grid cells ofthe generated map, wherein the one or more unknown grid cells are gridcells for which there is insufficient sensor information.

In Example 24, the subject matter of Example 23, wherein the executedinstructions causing the one or more processors to improve completenessof at least the portion of the generated map can further include theexecuted instructions causing the one or more processors to: eliminateunknown grid cells that are members of the determined prediction space.

In Example 25, the subject matter of Example 24, wherein the executedinstructions causing the one or more processors to eliminate unknowngrid cells can include the executed instructions causing the one or moreprocessors to assign a known status to the eliminated unknown gridcells.

In Example 26, the subject matter of any of Examples 23 to 25, whereinthe executed instructions causing the one or more processors todetermine coverage completeness can further include the executedinstructions causing the one or more processors to determine one or moreknown grid cells of the generated map comprises, wherein a grid cell isknown in response to determining from the obtained sensor data that thegrid cell is occupied or unoccupied by an object.

In Example 27, the subject matter of any of Examples 23 to 26, whereinone or more grid cells may be determined to be unknown in response todetermining the one or more grid cells are not covered by the sensorfield.

In Example 28, the subject matter of any of Examples 23 to 27, whereinone or more grid cells may be determined to be unknown in response todetermining the one or more grid cells are not covered due to failure ofone or more of the plurality of sensors.

In Example 29, the subject matter of any of Examples 23 to 28, whereinone or more grid cells may be determined to be unknown in response todetermining the one or more grid cells are not covered due to occlusion.

In Example 30, the subject matter of any of Examples 19 to 29, whereinthe object may be a vehicle.

In Example 31, the subject matter of any of Examples 18 to 30, whereinthe executed instructions may further cause the one or more processorsto obtain an area of interest, wherein the at least one portion of thegenerated map is the obtained area of interest.

In Example 32, the subject matter of any of Examples 18 to 31, whereinthe executed instructions can further cause the one or more processorsto determine one or more completeness metrics of the at least oneportion of the generated after updating the completeness.

In Example 33, the subject matter of any of Examples 19 to 32, whereinthe executed instructions causing the one or more processors todetermine object completeness comprises can further include the executedinstructions causing the one or more processors to determine no objectsare missing from the at least portion of the generated map based on thetracked objects, the determined amount of objects, and a net change ofobjects in the at least one portion of the generated map

In Example 34, the subject matter of Example 33, wherein in response todetermining no objects are missing, the executed instructions may causethe one or more processors to determine all grid cells of the least oneportion are known.

Example 35 is a system including one or more sensors; one or morecomputing devices, wherein the one or more computing devices areconfigured to: obtain sensor data from a plurality of sensors over time,the plurality of sensors covering a sensor field; generate a map byfusing the obtained sensor data, wherein the generated map comprises aplurality of grid cells at least partially covered by the sensor field;determine, using the obtained sensor data, completeness of at least oneportion of the generated map by determining object completeness andcoverage completeness of the map from the obtained sensor data; andupdate completeness of at least the portion of the generated map byreducing amount of unknown areas of the at least one portion of thegenerated map using the determined object completeness and thedetermined coverage completeness.

In Example 36, the subject matter of Example 35, which may furtherinclude one or more ingress and egress sensors, and wherein the one ormore computing devices may be further configured to: obtain sensor datafrom ingress and egress sensors; determine, from data obtained fromingress and egress sensors, objects entering and leaving at least oneportion of the generated map; determine, from the obtained sensor data,a current quantity of objects in the at least one portion of thegenerated map; and track, from the obtained sensor data, each of one ormore objects in the at least one portion of the generated map.

In Example 37, the subject matter of Example 36, wherein the one or morecomputing devices may be further configured to determine to objectcompleteness by determining disappearance of one or more objects fromthe at least portion of the generated map based on the tracked objects,the determined amount of objects, and a net change of objects in the atleast one portion of the generated map.

In Example 38, the subject matter of Example 37, wherein in response todetermining one or more objects have disappeared from the at least oneportion of the generated map, the one or more computing devices may befurther configured to determine a prediction space, wherein theprediction space comprises a set of grid cells for which the one or moredetermined disappeared objects can currently occupy.

In Example 39, the subject matter of Example 38, wherein the one or morecomputing devices configured to determine the prediction space canfurther include: for each determined disappeared object, the one or morecomputing devices being configured determining possible positions thedisappearing object can currently occupy using past sensor data of eachof the disappeared objects.

In Example 40, the subject matter of Example 39, wherein the one or morecomputing devices determining coverage completeness can include the oneor more computing devices determining one or more unknown grid cells ofthe generated map, wherein the one or more unknown grid cells are gridcells for which there is insufficient sensor information.

In Example 41, the subject matter of Example 40, wherein the one or morecomputing devices configured to update completeness of at least theportion of the generated map can include the one or more computingdevices being further configured to: eliminate unknown grid cells thatare members of the determined prediction space.

In Example 42, the subject matter of Example 41, wherein one or morecomputing devices configured to eliminate unknown grid cells can includethe one or computing devices being configured to reassign a known statusto the eliminated unknown grid cells.

In Example 43, the subject matter of any of Examples 41 or 42, whereinthe one or more computing devices configured to determine coveragecompleteness can further include the one or more computing devices beingfurther configured to determine one or more known grid cells of thegenerated map, wherein a grid cell is known in response to determiningfrom the obtained sensor data that the grid cell is occupied orunoccupied by an object.

In Example 44, the subject matter of any of Examples 41 to 43, whereinthe one or more computing devices are configured to determine one ormore grid cells as unknown in response to determining the one or moregrid cells are not covered by the sensor field.

In Example 45, the subject matter of any of Examples 41 to 44, whereinthe one or more computing devices are configured to determine one ormore grid cells as unknown in response to determining the one or moregrid cells are not covered due to failure of one or more of theplurality of sensors.

In Example 46, the subject matter of any of Examples 41 to 45, whereinthe one or more computing devices are configured to determine one ormore grid cells as unknown in response to determining the one or moregrid cells are not covered due to occlusion.

In Example 47, the subject matter of any of Examples 36 to 46, whereinthe object may be a vehicle.

In Example 48, the subject matter of any of Examples 35 to 47, whereinthe one or more computing devices may be further configured to obtain anarea of interest, wherein the at least one portion of the generated mapis the obtained area of interest.

In Example 49, the subject matter of any of Examples 35 to 48, whereinthe one or more computing devices may be further configured to determineone or more completeness metrics of the at least one portion of thegenerated after improving the completeness.

In Example 50, the subject matter of any of Examples 36 to 49, whereinthe one or more computing devices configured to determine objectcompleteness can include the one or more computing devices being furtherconfigured to determine that no objects are missing from the at leastportion of the generated map based on the tracked objects, thedetermined amount of objects, and a net change of objects in the atleast one portion of the generated map.

In Example 51, the subject matter of Example 50, wherein the one or morecomputing devices may be configured to, in response to determining noobjects are missing, determine all grid cells of the least one portionare known.

It should be noted that one or more of the features of any of theexamples above may be suitably combined with any one of the otherexamples.

The foregoing description has been given by way of example only and itwill be appreciated by those skilled in the art that modifications maybe made without departing from the broader spirit or scope of theinvention as set forth in the claims. The specification and drawings aretherefore to be regarded in an illustrative sense rather than arestrictive sense.

The scope of the disclosure is thus indicated by the appended claims andall changes which come within the meaning and range of equivalency ofthe claims are therefore intended to be embraced.

While the invention has been particularly shown, and described withreference to specific embodiments, it should be understood by thoseskilled in the art that various changes in form and detail may be madetherein without departing from the spirit and scope of the invention asdefined by the appended claims. The scope of the invention is thusindicated by the appended claims and all changes which come within themeaning and range of equivalency of the claims are therefore intended tobe embraced.

The invention claimed is:
 1. A method for improving map completeness,comprising: obtaining sensor data from a plurality of sensors over time,the plurality of sensors covering a sensor field; generating a map byfusing the obtained sensor data, wherein the generated map comprises aplurality of grid cells at least partially covered by the sensor field;determining, using the obtained sensor data, completeness of at leastone portion of the generated map by determining object completeness andcoverage completeness of the map from the obtained sensor data; andupdating completeness of at least the portion of the generated map byreducing an amount of unknown areas of the at least one portion of thegenerated map using the determined object completeness and thedetermined coverage completeness, wherein determining coveragecompleteness comprises determining one or more unknown grid cells of thegenerated map, wherein the one or more unknown grid cells are grid cellsfor which there is insufficient sensor information.
 2. The method ofclaim 1, wherein determining object completeness comprises: obtainingsensor data from ingress and egress sensors; determining, from dataobtained from ingress and egress sensors, objects entering and leavingat least one portion of the generated map; determining, from theobtained sensor data, a current quantity of objects in the at least oneportion of the generated map; and tracking, from the obtained sensordata, each of one or more objects in the at least one portion of thegenerated map.
 3. The method of claim 2, wherein determining objectcompleteness comprises determining disappearance of one or more objectsfrom the at least portion of the generated map based on the tracking,the determined amount of objects and a net change of objects in the atleast one portion of the generated map.
 4. The method of claim 3,wherein in response to determining one or more objects have disappearedfrom the at least one portion of the generated map, the method furthercomprises determining a prediction space, wherein the prediction spacecomprises a set of grid cells for which the one or more determineddisappeared objects can currently occupy.
 5. The method of claim 4,wherein determining the prediction space further comprises: for eachdetermined disappeared object, determining possible positions thedisappearing object can currently occupy using past sensor data of eachof the disappeared objects.
 6. The method of claim 5, wherein updatingcompleteness of at least the portion of the generated map comprises:eliminating unknown grid cells that are members of the determinedprediction space.
 7. The method of claim 6, wherein eliminating unknowngrid cells comprises assigning a known status to the eliminated unknowngrid cells.
 8. The method of claim 5, wherein determining coveragecompleteness further comprises determining one or more known grid cellsof the generated map, wherein a grid cell is known in response todetermining from the obtained sensor data that the grid cell is occupiedor unoccupied by an object.
 9. The method of claim 1, wherein one ormore grid cells are determined to be unknown in response to determiningthe one or more grid cells are not covered by the sensor field.
 10. Themethod of claim 1, wherein one or more grid cells are determined to beunknown in response to determining the one or more grid cells are notcovered due to failure of one or more of the plurality of sensors. 11.The method of claim 1, wherein one or more grid cells are determined tobe unknown in response to determining the one or more grid cells are notcovered due to occlusion.
 12. The method of claim 2, wherein the objectis a vehicle.
 13. The method of claim 1, further comprising obtaining anarea of interest, wherein the at least one portion of the generated mapis the obtained area of interest.
 14. The method of claim 1, furthercomprising, determining one or more completeness metrics of the at leastone portion of the generated after improving the completeness.
 15. Themethod of claim 2, wherein determining object completeness comprisesdetermining no objects are missing from the at least portion of thegenerated map based on the tracking, the determined amount of objectsand a net change of objects in the at least one portion of the generatedmap.
 16. The method of claim 15, wherein in response to determining noobjects are missing, determining all grid cells of the least one portionare known.
 17. One or more computing devices comprising one or moreprocessors and at least one non-transitory computer-readable storagemedium including instructions that, when executed by the one or moreprocessors, cause the one or more processors to: obtain sensor data froma plurality of sensors over time, the plurality of sensors covering asensor field; generate a map by fusing the obtained sensor data, whereinthe generated map comprises a plurality of grid cells at least partiallycovered by the sensor field; determine, using the obtained sensor data,completeness of at least one portion of the generated map by determiningobject completeness and coverage completeness of the map from theobtained sensor data; and update completeness of at least the portion ofthe generated map by reducing an amount of unknown areas of the at leastone portion of the generated map using the determined objectcompleteness and the determined coverage completeness, wherein todetermine coverage completeness comprises to determine one or moreunknown grid cells of the generated map, wherein the one or more unknowngrid cells are grid cells for which there is insufficient sensorinformation.
 18. The one or more computing devices of claim 17, whereinthe executed instructions cause the one or more processors to determineto object completeness by causing the one or more processors to: obtainsensor data from ingress and egress sensors; determine, from dataobtained from ingress and egress sensors, objects entering and leavingat least one portion of the generated map; determine, from the obtainedsensor data, a current quantity of objects in the at least one portionof the generated map; and track, from the obtained sensor data, each ofone or more objects in the at least one portion of the generated map.19. The one or more computing devices of claim 18, wherein the executedinstructions cause the one or more processors to determine to objectcompleteness by causing the one or more processors to determinedisappearance of one or more objects from the at least portion of thegenerated map based on the tracked objects, the determined amount ofobjects, and a net change of objects in the at least one portion of thegenerated map.